Repository navigation
Conversation
createBounded allocated one bucket per entry of its cap, up front and for good, since the table never resizes. A table capped at 131,072 entries retained a 512 KiB array even while nearly empty, which on a small heap is out of proportion and, under G1, a humongous allocation. Size bounded tables by bucketCountFor(maxCapacity) instead: one bucket per entry for small tables, but at most 1/1024 of the maximum heap and below G1's humongous threshold, with a floor of 1,024 buckets. The cap is unchanged; chains carry any excess. A new overload takes an explicit bucket count for callers that know their expected population. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each flag evaluation used to build a FlagEvalEvent and offer it to a bounded 4,096-slot queue, which a single worker thread drained into the aggregator. Under load the worker could not keep up, and the queue dropped 30-67% of evaluations. Record evaluations straight into a ConcurrentFlagAggregator instead. An evaluation that lands in an existing bucket costs a lock-free chain scan and a few atomic updates and allocates nothing; a new bucket is inserted under the toolbox ConcurrentHashtable's write lock, which also enforces the tier caps. The tiers, caps and bucket keys are unchanged, so the payloads are the same. Buckets are drained in place: a bucket whose count goes from zero to one pushes itself onto a lock-free stack, and the flush reads and resets the counters of the buckets on it. An evaluation racing the flush is counted in this flush or the next one, never lost. Buckets idle for a whole window are swept out, freeing their slots under the caps. The worker now only flushes, every flush interval and at shutdown. Evaluations recorded after close are still counted as closed drops, and close() counts those that land after the final flush. The queue-overflow drop metric goes away with the queue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LLM ValidationLLM Validation Gate — java-agent-instructions✅ PASS
6 case(s) · quality 81.2 → 81.2 (+0.1) · win-rate 49% [46%–51%] Details — analysis and per-case resultsAnalysisChanged instruction file(s): No safety or blocking-case regressions across 6 case(s). Overall pairwise win-rate 49% [46%–51%], quality +0.1 — see the verdict above for whether that clears the noise band. Results
Cases
Per-dimension scores, token usage, latency, and estimated cost are in the CI job logs. |
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Stacked on #13609. It also carries the commit of #13612 (heap-aware
ConcurrentHashtablebucket sizing), which drops out once that lands.Flag evaluations are now aggregated on the evaluating thread, into toolbox
ConcurrentHashtables. This replaces allocating aFlagEvalEventper evaluation and queueing it for a single worker thread. It's the only path; there's no switch.Recording: an evaluation that lands in an existing bucket costs a lock-free chain scan and a few atomic updates, and allocates nothing:
A new bucket is inserted through a
Reservationunder the table's write lock, which also enforces the tier caps. The tiers, caps and bucket keys are unchanged (full tier, per-flag cap, degraded tier with the consent fold), so the payloads don't change.Draining in place: buckets live across flushes. A bucket whose count goes from 0 to 1 pushes itself onto a lock-free stack through its own
dirtyNextfield. The flush takes the stack and reads and resets each bucket's counters. An evaluation racing the flush is counted in this flush or the next one, never lost. Buckets idle for a whole window are swept out, freeing their slots under the caps.The worker only flushes, every flush interval and at shutdown. It waits on the shutdown latch rather than polling a queue. Evaluations recorded after close are counted as closed drops.
Deleted:
FlagEvalEvent,FlagEvaluationAggregator, the evaluationEventQueueand its overflow metric, andFlagEvaluationEnqueueContentionBenchmark.EventQueuestays for exposures.Motivation
Under load, the single worker couldn't keep up with the queue: the unpaced client benchmark dropped 29–67% of flag-evaluation events. The ConcurrentHashtable works as a queue that merges same-key events on arrival, so its size depends on the number of distinct rows, not on the event rate.
A prototype of this design was measured on the FFE roll-up by a benchmark worker (
ClientEvaluationBenchmark, JDK 21, one jar, queue path against caller-side, 3 forks each):Equal work (paced, 4 threads, the queue dropping about 1%), where the time per evaluation includes the pacing delay, so only the difference counts:
flagssplitsvariationsDrops: 0.0% in every unpaced fork, against 29–67% on the queue path. Unpaced timings aren't comparable, because a dropped evaluation skips the hook's work.
Exact counts: at 8 threads, the posted
evaluation_countmatched the evaluations performed: 223,638,646 = 223,638,646 forflags, andsplitsandvariationsmatched too. The queue path posted 68.8M of 213.1M.128 MB G1 heap:
flagsthroughput +17%, and GC pause time per million evaluations 24.4 → 12.8 ms.Hot path: the aggregator is 4.8–6.9% of evaluating-thread time, with no contention frames. A flush takes about 100 ms per 10 s window.
Additional Notes
dougqh/ffe-caller-aggregation-v2). A run of this branch against Key aggregator contexts by an order-independent hash (FFE optimization) #13609 is queued, and its results will be added here.ConcurrentFlagAggregatorTest, including a concurrency test that drains while producers run and loses nothing. The pipeline, hook and payload tests are ported to the new path.🤖 Generated with Claude Code